Implement partial success logging in OTLP exporters by michaelsafyan · Pull Request #4805 · open-telemetry/opentelemetry-python

michaelsafyan · 2025-11-11T08:48:37Z

Log partial success

Partial successes during OTLP export is now logged.

Fixes #4803

Type of change

New feature (non-breaking change which adds functionality)
[?] This change requires a documentation update

How Has This Been Tested?

Added new unit tests covering the relevant cases.

Does This PR Require a Contrib Repo Change?

No.

Checklist:

Followed the style guidelines of this project
Changelogs have been updated
Unit tests have been added
[] Documentation has been updated - unsure what to update

…ging

…levels to avoid changing default behaviors. Apply test and formatting fixes.

This appears to cause an issue with "tox -e docs".

DylanRussell

Why put this log behind an env var ? Why not just always log when the field is set -- I'm assuming it's only set when there was a partial success -- it doesn't sound too noisy to me.

michaelsafyan · 2025-11-12T22:31:33Z

Why put this log behind an env var ? Why not just always log when the field is set -- I'm assuming it's only set when there was a partial success -- it doesn't sound too noisy to me.

I'm uncertain as to the level of noisiness, and so I wanted to minimize disruption to existing users. I could imagine it being somewhat annoying if every batch included one persistently malformed span that just ended up creating log spam.

A more sophisticated approach might be to opt-in to recording every partial success, while elevating the verbosity and logging the partial success unconditionally if every record was a failure (if num_rejected_{type} matches the number of items in the batch). But I wanted to keep things relatively simple.

DylanRussell · 2025-11-14T17:16:34Z

We already have the DuplicateFilter on this logger, maybe we should tweak that to de-dupe based on line no and module instead of message which can be unique so that it filters stuff out better.

I think it's a bit weird to put just 1 log message behind this env var

michaelsafyan · 2025-11-20T03:21:23Z

Thanks for the review, @DylanRussell . I updated the logic to no longer make the logging conditioned on OTEL_LOG_LEVEL. I'm not sure what the DuplicateFilter is you are referring to ... are you asking to also remove the special casing for logs (because you are not concerned about partial success causing a loop?)

DylanRussell · 2025-11-20T17:10:38Z

    ) -> ExportServiceRequestT:
        pass

+    def _log_partial_success(self, partial_success):


nit: can we just inline this code instead of having a function (same thing below)

DylanRussell · 2025-11-20T17:20:59Z

We add a DuplicateFilter to the logger on line 116 (as an aside I don't like the github code review tool, which doesn't let me add a comment to a line that wasn't modified...)

I'm thinking we should update that to reference record.lineno instead of record.msg which could be unique.... and that is sufficient to suppress noisy logging, and also avoid the endless logging issues...

If you want to take a stab at that change here that'd be great, if not I think we can proceed without that and I can make it separately..

I don't think we should special case the logging exporter because we have this duplicate filter thing.

Thanks, @DylanRussell ! Updated accordingly. Please take another look.

Also, any idea what's going on with the failed tests? They look to be unrelated to this change:

That's some sort of flake we get all the time (can't find a commit hash), IDK why we get that. It's OK to ignore

…ging

…lly.

DylanRussell · 2025-11-25T14:25:23Z

    ) -> ExportServiceRequestT:
        pass

+    def _process_response(self, response):


nit: can we inline this code instead ? IMO it's more readable

DylanRussell · 2025-11-25T14:42:03Z

@@ -46,17 +46,36 @@ class DuplicateFilter(logging.Filter):
    recursion depth exceeded issue in cases where logging itself results in an exception."""

    def filter(self, record):


IMO I would simplify all this and just have the logic as:

current_log_by_line = ( record.module, record.pathname, record.lineno, time_boundary, ) previous_log_by_line = getattr(self, "last_source", None) keep_processing = current_log_by_line != previous_log_by_line self.last_source = current_log_by_line return keep_processing

Although I'm realizing now that we should store the previous 10 previous_log_by_line in like a deque or OrderedDict instead.. Right now if duplicate log messages are interleaved we will not filter anything out

Also we have tests for this thing here:

opentelemetry-python/opentelemetry-sdk/tests/shared_internal/test_batch_processor.py

Lines 259 to 266 in 62da90e

class TestCommonFuncs(unittest.TestCase):

def test_duplicate_logs_filter_works(self):

test_logger = logging.getLogger("testLogger")

test_logger.addFilter(DuplicateFilter())

with self.assertLogs("testLogger") as cm:

test_logger.info("message")

test_logger.info("message")

self.assertEqual(len(cm.output), 1)

DylanRussell · 2025-11-25T14:50:06Z

 # See the License for the specific language governing permissions and
 # limitations under the License.

+


nit: can revert this

DylanRussell · 2025-11-25T14:50:25Z

@@ -11,6 +11,7 @@
 # See the License for the specific language governing permissions and


You could also update the HTTP exporters the same way I think ? https://github.com/open-telemetry/opentelemetry-python/tree/main/exporter/opentelemetry-exporter-otlp-proto-http/src/opentelemetry/exporter/otlp/proto/http

github-actions · 2026-03-04T03:55:31Z

This PR has been automatically marked as stale because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 days of this comment.
If you're still working on this, please add a comment or push new commits.

github-actions · 2026-03-18T04:06:23Z

This PR has been closed due to inactivity. Please reopen if you would like to continue working on it.

Implement partial success logging.

1008906

michaelsafyan mentioned this pull request Nov 11, 2025

Make it possible to opt-in to logging partial errors #4803

Open

michaelsafyan added 3 commits November 11, 2025 03:08

Add tests.

72b2325

Merge branch 'open-telemetry:main' into issue4803_partial_success_log…

baefbc0

…ging

Fix: OTEL_LOG_LEVEL default to info for partial success logging.

e008658

michaelsafyan marked this pull request as ready for review November 11, 2025 20:55

michaelsafyan requested a review from a team as a code owner November 11, 2025 20:55

michaelsafyan changed the title ~~[DRAFT] Implement partial success logging to tackle issue 4803~~ Implement partial success logging in OTLP exporters Nov 11, 2025

michaelsafyan added 3 commits November 11, 2025 23:51

Fixes partial success logging to apply at only 'verbose' and 'debug' …

d40ab67

…levels to avoid changing default behaviors. Apply test and formatting fixes.

Simplify tests to address pylint issue and reformat with ruff.

5725ccd

Undo changes to environment variable docs

818456e

This appears to cause an issue with "tox -e docs".

DylanRussell reviewed Nov 12, 2025

View reviewed changes

Merge branch 'main' into issue4803_partial_success_logging

10aedae

michaelsafyan added 2 commits November 19, 2025 21:04

Merge branch 'main' into issue4803_partial_success_logging

b44507a

Remove conditional logic around recording partial success.

d84b22a

DylanRussell reviewed Nov 20, 2025

View reviewed changes

michaelsafyan added 2 commits November 24, 2025 20:58

Merge branch 'open-telemetry:main' into issue4803_partial_success_log…

11d3e09

…ging

Improve DuplicateFilter behavior and log partial success unconditiona…

a00c8bd

…lly.

DylanRussell reviewed Nov 25, 2025

View reviewed changes

tammy-baylis-swi added this to Python PR digest Feb 26, 2026

tammy-baylis-swi moved this to Reviewed PRs that need fixes in Python PR digest Feb 26, 2026

github-actions Bot added the Stale label Mar 4, 2026

github-actions Bot closed this Mar 18, 2026

github-project-automation Bot moved this from Reviewed PRs that need fixes to Done in Python PR digest Mar 18, 2026

AnkushMalaker mentioned this pull request Apr 14, 2026

fix(otel): surface silently-rejected spans via partial_success rungalileo/galileo-python#551

Draft

3 tasks

		@@ -46,17 +46,36 @@ class DuplicateFilter(logging.Filter):
		recursion depth exceeded issue in cases where logging itself results in an exception."""

		def filter(self, record):

	class TestCommonFuncs(unittest.TestCase):
	def test_duplicate_logs_filter_works(self):
	test_logger = logging.getLogger("testLogger")
	test_logger.addFilter(DuplicateFilter())
	with self.assertLogs("testLogger") as cm:
	test_logger.info("message")
	test_logger.info("message")
	self.assertEqual(len(cm.output), 1)

		# See the License for the specific language governing permissions and
		# limitations under the License.

		@@ -11,6 +11,7 @@
		# See the License for the specific language governing permissions and

Conversation

michaelsafyan commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Log partial success

Type of change

How Has This Been Tested?

Does This PR Require a Contrib Repo Change?

Checklist:

Uh oh!

DylanRussell left a comment

Choose a reason for hiding this comment

Uh oh!

michaelsafyan commented Nov 12, 2025

Uh oh!

DylanRussell commented Nov 14, 2025

Uh oh!

michaelsafyan commented Nov 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Mar 4, 2026

Uh oh!

github-actions Bot commented Mar 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

michaelsafyan commented Nov 11, 2025 •

edited

Loading